Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable Automation style tests of v2 applications - Fluent assertions #3993

Open
wants to merge 26 commits into
base: v2_develop
Choose a base branch
from
Open

Enable Automation style tests of v2 applications - Fluent assertions #3993

wants to merge 26 commits into from
+1,028 −6

Conversation

tznind
Copy link
Collaborator

@tznind tznind commented Mar 16, 2025 •
edited
Loading

Feature

Enable easy end to end testing of v2 applications. Initially for just ourselves but later perhaps our users too.

  • Standalone project with only dependency on Terminal.Gui
  • Not tied to xunit / nunit for maximum flexibility
  • Test both drivers simultaneously
    [Theory]
    [InlineData (V2TestDriver.V2Win)]
    [InlineData(V2TestDriver.V2Net)]
    public void GuiTestContext_StartsAndStopsWithoutError (V2TestDriver d)
    {
        using var context = With.A<Window> (40, 10,d);
  • Screenshot and Logging support
using GuiTestContext c = With.A<Window> (40, 10,d)
                             .WithContextMenu (ctx, menuItems)
                             .ScreenShot ("Before open menu", _out)

                             // Click in main area inside border
                             .RightClick (1, 1)
                             .ScreenShot ("After open menu", _out)
                             .Down ()
                             .Down ()
                             .Down ()
                             .Right()
                             .ScreenShot ("After open submenu", _out)
                             .Down ()
                             .Enter ()
                             .ScreenShot ("Menu should be closed after selecting", _out)
                             .Stop ()
                             .WriteOutLogs (_out);

    Assert.True (clicked);
Example Output 

Before open menu:
┌──────────────────────────────────────┐
│                                      │
│                                      │
│                                      │
│                                      │
│                                      │
│                                      │
│                                      │
│                                      │
└──────────────────────────────────────┘

After open menu:
┌──────────────────────────────────────┐
│                                      │
│┌────────┐                            │
││ One    │                            │
││ Two    │                            │
││ Three  │                            │
││ Four  ►│                            │
││ Five   │                            │
││ Six    │                            │
└└────────┘────────────────────────────┘

After open submenu:
┌──────────────────────────────────────┐
│                                      │
│┌────────┐┌───────────┐               │
││ One    ││ SubMenu1  │               │
││ Two    ││ SubMenu2  │               │
││ Three  ││ SubMenu3  │               │
││ Four  ►││ SubMenu4  │               │
││ Five   ││ SubMenu5  │               │
││ Six    ││ SubMenu6  │               │
└└────────┘│ SubMenu7  │───────────────┘

Menu should be closed after selecting:
┌──────────────────────────────────────┐
│                                      │
│                                      │
│                                      │
│                                      │
│                                      │
│                                      │
│                                      │
│                                      │
└──────────────────────────────────────┘

Main Loop Coordinator booting...
Main Loop Coordinator booting complete
[ConfigurationManager] [Load] reset = False
[ConfigurationManager] [Reset] _allConfigProperties = 
[ConfigurationManager] [Initialize]   Class: TestConfiguration
[ConfigurationManager] [Initialize]   Class: AppSettingsTestClass
[ConfigurationManager] [Initialize]   Class: UICatalogApp
[ConfigurationManager] [Initialize]   Class: ConfigurationEditor
[ConfigurationManager] [Initialize]   Class: Application
[ConfigurationManager] [Initialize]   Class: ConfigurationManager
[ConfigurationManager] [Initialize]   Class: ThemeManager
[ConfigurationManager] [Initialize]   Class: Color
[ConfigurationManager] [Initialize]   Class: Colors
[ConfigurationManager] [Initialize]   Class: Glyphs
[ConfigurationManager] [Initialize]   Class: FileDialogStyle
[ConfigurationManager] [Initialize]   Class: Key
[ConfigurationManager] [Initialize]   Class: Button
[ConfigurationManager] [Initialize]   Class: CheckBox
[ConfigurationManager] [Initialize]   Class: Dialog
[ConfigurationManager] [Initialize]   Class: FileDialog
[ConfigurationManager] [Initialize]   Class: FrameView
[ConfigurationManager] [Initialize]   Class: ContextMenu
[ConfigurationManager] [Initialize]   Class: MessageBox
[ConfigurationManager] [Initialize]   Class: NerdFonts
[ConfigurationManager] [Initialize]   Class: Window
[SettingsScope] [Update] Read from "resource://[Terminal.Gui]/Terminal.Gui.Resources.config.json"
[SettingsScope] [Update] Read from "ConfigurationManager.RuntimeConfig"
[SettingsScope] [Update] "./.tui/config.json" does not exist.
[SettingsScope] [Update] "C:\Users\44777/.tui/config.json" does not exist.
[SettingsScope] [Update] "./.tui/ReSharperTestRunner.config.json" does not exist.
[SettingsScope] [Update] "C:\Users\44777/.tui/ReSharperTestRunner.config.json" does not exist.
Run 'Window(){X=0,Y=0,Width=0,Height=0}'
[MainLoop] [AnySubViewsNeedDrawn] Window triggered redraw (NeedsDraw=True NeedsLayout=True) 
Console size changes from '{Width=0, Height=0}' to {Width=40, Height=10}
[InputProcessor] [OnMouseEvent] Mouse Interpreter raising Button3Pressed, ReportMousePosition
[InputProcessor] [OnMouseEvent] Mouse Interpreter raising Button3Released, ReportMousePosition
[MouseInterpreter] [RaiseClick] Raising click event:Button3Clicked at screen {X=1,Y=1}
[InputProcessor] [OnMouseEvent] Mouse Interpreter raising Button3Clicked
[MainLoop] [AnySubViewsNeedDrawn] Window triggered redraw (NeedsDraw=True NeedsLayout=True) 
[AnsiResponseParser] [AppendOutput] AnsiResponseParser releasing '\0'
[InputProcessor] [OnKeyDown] InputProcessor raised CursorDown
[MainLoop] [AnySubViewsNeedDrawn] Menu triggered redraw (NeedsDraw=True NeedsLayout=False) 
[AnsiResponseParser] [AppendOutput] AnsiResponseParser releasing '\0'
[InputProcessor] [OnKeyDown] InputProcessor raised CursorDown
[MainLoop] [AnySubViewsNeedDrawn] Menu triggered redraw (NeedsDraw=True NeedsLayout=False) 
[AnsiResponseParser] [AppendOutput] AnsiResponseParser releasing '\0'
[InputProcessor] [OnKeyDown] InputProcessor raised CursorDown
[MainLoop] [AnySubViewsNeedDrawn] Menu triggered redraw (NeedsDraw=True NeedsLayout=False) 
[AnsiResponseParser] [AppendOutput] AnsiResponseParser releasing '\0'
[InputProcessor] [OnKeyDown] InputProcessor raised CursorRight
[MainLoop] [AnySubViewsNeedDrawn] Window triggered redraw (NeedsDraw=True NeedsLayout=True) 
[AnsiResponseParser] [AppendOutput] AnsiResponseParser releasing '\0'
[InputProcessor] [OnKeyDown] InputProcessor raised CursorDown
[MainLoop] [AnySubViewsNeedDrawn] Menu triggered redraw (NeedsDraw=True NeedsLayout=False) 
[AnsiResponseParser] [AppendOutput] AnsiResponseParser releasing '
'
[InputProcessor] [OnKeyDown] InputProcessor raised Enter
[MainLoop] [AnySubViewsNeedDrawn] Window triggered redraw (NeedsDraw=True NeedsLayout=True) 
RequestStop ''
Input loop exited cleanly

Goals

Perform 'automation' style tests using black box input/output.

  • Design from ground up to prevent tests ever hanging (i.e. with timeouts etc baked in)
  • Simplify API so user doesn't need to know as much about the internals of Terminal.Gui to test things (e.g. no need to call NewKeyEvent etc)
  • Traceability (comprehensive logging and 'screenshoting' to assist debugging failure states)
  • v2 drivers only but ideally support net and win
  • Input events supplied at raw 'console' level i.e. going through whole application/driver subsystem.

Examples

Started exploring a way to empower our users and ourselves to write cleaner tests around full Terminal.Gui application loops.

    [Fact]
    public void TestWindowsResize ()
    {
        var lbl = new Label ()
                                {
                                    Width = Dim.Fill ()
                                };
        using var c = With.A<Window> (40, 10)
                          .Add (lbl )
                          .Assert (lbl.Frame.Width.Should().Be(38)) // Window has 2 border
                          .ResizeConsole (20,20)
                          .Assert (lbl.Frame.Width.Should ().Be (18))
                          .Stop ();

    }

Also

    [Fact]
    public void ContextMenu_Click ()
    {
        var clicked = false;

        var ctx = new ContextMenu ();
        var menuItems = new MenuBarItem (
                                         [
                                             new ("_New File", string.Empty, () => { clicked = true; })
                                         ]
                                        );

        using var c = With.A<Window> (40, 10)
                          .WithContextMenu(ctx,menuItems)
                          // Click in main area inside border
                          .RightClick(1,1)
                          .ScreenShot ("After open menu",_out)
                          .LeftClick (3, 3)
                          .Stop ();
        Assert.True (clicked);
    }

Produces


  Standard Output: 
After open menu:
┌──────────────────────────────────────┐
│                                      │
│┌───────────┐                         │
││ New File  │                         │
│└───────────┘                         │
│                                      │
│                                      │
│                                      │
│                                      │
└──────────────────────────────────────┘

Class Diagram

image

Fixes

  • Fixes #_____

Proposed Changes/Todos

  • Todo 1

Pull Request checklist:

  • I've named my PR in the form of "Fixes #issue. Terse description."
  • My code follows the style guidelines of Terminal.Gui - if you use Visual Studio, hit CTRL-K-D to automatically reformat your files before committing.
  • My code follows the Terminal.Gui library design guidelines
  • I ran dotnet test before commit
  • I have made corresponding changes to the API documentation (using /// style comments)
  • My changes generate no new warnings
  • I have checked my code and corrected any poor grammar or misspellings
  • I conducted basic QA to assure all features are working

@TheTonttu
Copy link
Contributor

FYI there are some license related controversy surrounding Fluent Assertions. 7.x was the last Apache licensed version before the library switched to custom hybrid license. It doesn't really affect this project but just something to keep in mind if you're planning on relying on it extensively.

@tznind
Copy link
Collaborator Author

tznind commented Mar 16, 2025

Interesting, in that case maybe I will just switch to Action and let caller decide. Thanks for the heads up!

@tznind
Copy link
Collaborator Author

tznind commented Mar 16, 2025 •
edited
Loading

I've chopped out fluent assertions so we can stick with MIT license and renamed the new project FluentTesting instead to avoid any confusion.

@tznind tznind mentioned this pull request Mar 17, 2025
Open
@tznind tznind marked this pull request as ready for review March 20, 2025 08:42
@tznind tznind requested a review from tig as a code owner March 20, 2025 08:42
@tznind tznind changed the title Fluent assertions Enable Automation style tests of v2 applications - Fluent assertions Mar 20, 2025
@tznind
Copy link
Collaborator Author

tznind commented Mar 20, 2025

Ok I think this is good as a first cut!

Let me know what you think, to start with this can just be for internal testing. Later on we can probably setup CI to release it as a new nuget package to enable our users to test their applications.

Currently the faked input support is:

  • Cursor keys
  • Mouse left/right click

We can add other things later on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tig tig Awaiting requested review from tig tig is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@tznind @TheTonttu @BDisp @tig